In gaming systems which work with stationary displays like televisions and computer screens, a user often controls action of an object on the display with a real controller device. Besides games, other applications use real controllers as well for controlling some action of an object on the stationary display in the context of an executing application. Although the realism of the stationary display is less than in an augmented reality experience, many users enjoy the touch sensation of using a real controller which directs action.